On Nearest Neighbor Classification Using Adaptive Choice of k

Nearest neighbor classification is one of the simplest and popular methods for statistical pattern recognition. It classifies an observation x to the class, which is the most frequent in the neighborhood of x. The size of this neighborhood is usually determined by a predefined parameter k. Normally, one uses cross-validation techniques to estimate the optimum value of this parameter, and that estimated value is used for classifying all observations. However, in classification problems, in addition to depending on the training sample, a good choice of k depends on the specific observation to be classified. Therefore, instead of using a fixed value of k over the entire measurement space, a spatially adaptive choice of k may be more useful in practice. This article presents one such adaptive nearest neighbor classification technique, where the value of k is selected depending on the distribution of competing classes in the vicinity of the observation to be classified. The utility of the proposed method has been illustrated using some simulated examples and well-known benchmark datasets. Asymptotic optimality of its misclassification rate has been derived under appropriate regularity conditions.